Alternative ways of monitoring collaboration

Teachers of networked collaborative classrooms, with multiple groups interacting in parallel, need assistance for better understanding and regulating the learning process. They may use tools that collect and process students’ activity data, and generate views of students’ activities. Classroom management tools may be used to observe and control students’ screens, but often fail to exploit interaction data. We present the results of a study where teachers used a classroom management environment to monitor students’ collaborative activities, and a discussion of the alternative views provided by a cscl tool, designed for teacher’s support. Monitoring Groups’ Collaborative activities: A task for the teacher In CSCL research community there is an increasing interest in the design of tools that support students’ computer mediated collaborative learning. Tool designers build tools for students and teachers; however they seem to pay more attention on students’ needs than on teachers’ (Dimitracopoulou 2005). Examples of teacher tools include Prof-CHENE with a communication space for the teacher, that was used to investigate, the nature of knowledge of teachers on students collaborative problem solving (Baker et al 2001). González and Suthers (2002) have proposed a pedagogical agent that helps students collaborate while solving Entity Relationship modeling problems in COLER. The generated advice, during online intervention, was based primarily on comparison between students' individual and group solutions and student participation (e.g. contributions to the group diagram). During Collaborative mediated collaboration (CMC) teachers play multiple roles: providers of information, managers of the students’ interaction, and moderators of the students’ debate (Baker et al, 2001). Dillenbourg (2005) estimates that the cost for regulation and tutoring multiple groups in a CSCL environment might be very high, so the teacher, like Boeing pilot, is expected to use a “cockpit”, for a quick and effective class and group overview. CSCL tools and class management tools have however different objectives. CSCL tools aim at facilitation of collaboration, while classroom management systems, based in network management infrastructure, aim at the management, observation and control of students working in a network either individually or in groups. In this paper, we report on a study that discusses the effectiveness of these two approaches. First a class monitoring environment has been used to support the teacher. Then a CSCL environment and the support that it provides to the teacher is discussed. At the final part of the paper, a comparison of the affordances of the two approaches is included, along with the future perspectives and implications for next generation monitoring tools. Context of the study A network based synchronous collaborative drawing tool, Synergo, has been used for a number of years in an undergraduate course, part of a Computer Engineering degree curriculum. A typical collaborative activity involved dyads of first year students, during a laboratory class and lasted 60-90 minutes approximately. Their joint task was to build a diagrammatic representation of an algorithm that was provided in textual form. Students interacted through Synergo where in a shared window they built their diagrams (i.e. flowchart), and communicated via a chat tool. Teachers observed students’ screens and intervened to regulate their activities, without additional support by any tool. Students were asked to seek their partners’ help during the problem solving process. Face to face interaction was permitted with restrictions, as students, and teachers were present in the same laboratory. Teacher interventions were permitted in case of major difficulties. Two teacher tools have been used to support teachers in similar settings in subsequent years: NetOp School TM (www.netop.com), a commercial Class Management Tool (CMT) and Synergo Supervisor, a tool specially designed for supervising and monitoring Synergo collaborative activities. The latter study took place with CMT in two subsequent weeks during last semester, in two laboratory sessions. Five laboratory groups involving 150 students, participated. The observations made are discussed next. A class management approach for collaborative class monitoring The class management tool used, NETOP School, include a module for instruction planning, class management and monitoring. The teacher is able to observe students activities at real time, control students actions as permitting or denying access to applications, or locking students workstation. The teacher has the ability to monitor the whole class using thumbnail views of student screens, record students’ screens or take snapshots of them. In our study, it was considered most appropriate to use the monitoring facility for observing the activity of 30% of the students of the group which have been selected randomly, i.e. for this study only 10 students’ activities were monitored per lab. One teacher and two assistants were present at each lab session. The class monitoring environment was used by the teacher. The teacher observed students screens in thumbnails, or the full size screen of a selected student. Figure 1. Class management system thumbnails overview. Based on the teacher’s report after each laboratory activity, we discuss the first findings. Students remained on task, so Synergo was the main application in use, therefore, monitoring of each individual screen was easy. However the teacher was able to monitor only the part of the shared working space and the chat dialogue that students were looking at. The persistency of the shared working space and chat, served as a shared working memory (Dillenbourg, 2005) for each group, but not for the teacher. For instance an empty working space (e.g. left bottom of fig. 1) may be a typical situation that requires intervention, as it indicates small progress. However after close observation for some minutes, the teacher discovered that this student had scrolled the already developed part of the solution out of the screen. In order to follow groups’ reasoning and reconstruct their activity (Baker et al, 2001) the teacher had to look at the whole process. This was not possible since only the current status of the solution and chat that students worked on, was available. Keeping track of each students’ solution and collaboration process by shifting attention among groups, even if only 30% of the students’ activities were monitored, was found particularly demanding. During this study a requirement of visualization of more abstract views of group interaction data was raised. In addition, asynchronous navigation of the history of the working space or chat of each group was discussed. The teacher was interested in solution process (Voyiatzaki et al , 2006). Thumbnails provided a comparative view of the progress of students’ work. E.g. in fig. 1 the teacher can see that the upper screen the group had proceeded with the implementation of the loop, while this is not the case in the lower screen. Concerning collaboration, this could be monitored only through inspecting the chat, however this was also hard, as students in these activities used elliptical language in chat messages, and they actually used two related means of communication: chat and flowchart symbols. Thus it was easier to monitor the status of problem solution at the level of the class and group and not at the individual level. Alarms for unexpected situations were not produced, as activity data could not be processed. Monitoring a collaborative class through Synergo Supervisor Synergo Supervisor is a tool designed to support the monitoring and supervising role of the teacher in these activities, and has been used in similar setting in previous courses (Voyiatzaki et al , 2008). In the spirit of Cockpit (Dillenbourg, 2005), the teacher has a quick overview of group’s activities and identification of the groups that do not function as expected, like the pilot who identifies faults in the plane. The teacher monitors each group’s process in various levels of detail, through graphs, video like representations, and sequences of snapshots of the drawing space, based on calculation and integration of logged data. The teacher is able to have an overview of the current status of the class activity. He can navigate asynchronously more detailed views of each group’s interaction. Changing focus, he may inspect the history of the process at student, group or class level. It offers an overview of the groups’ processes with comparative view of snapshots taken during solution process, and with graphical representation of students’ interaction. The teacher had an overview of groups’ activities in a class overview level (Fig 2 (1),(2)). Unexpected interactions could be identified: A warning symbol of a “lamp” and “chat” have been associated to groups 1 and 3 respectively, as their state was not as expected. Comparison of class and current group behaviour, activated triggers that generated these indicators. In the graphical overview representation (2) the teacher could see that one group was more active than the others or exchanged fewer chat messages. He focused, on groups’ dialogues (3), in order to detect difficulties or misunderstandings that might require intervention. He was always aware of groups’ interaction overview (2) and able to navigate asynchronously to current and previous snapshots of their solution . Selected group’s current solution instance was compared with other groups’ solutions (5), even in a less detailed view, to detect differences, for teacher’s intervention. The teacher monitored individual's contribution, either by content of messages (3) or by comparing group members’ interactions (4). The teacher was able neither to observe the monitor of each student, nor to take the control from the student (e.g. to deny access to other applications). Figure 2. SYNERGO Supervisor . Perspectives on tools for supporting teacher understanding From the discussion of the first outcomes of our study, it seems that very rich representation as students screens, are not always enough to support teacher’s understanding during collaborative activities. A combination of the two tools presents advantages: Synergo Supervisor may be used as teacher cockpit while class management system as teacher navigator, control and communication channel. We want to analyse the use of the two approaches by the teacher, to gain better and in depth understanding of teacher needs in real time class collaborative setting. So a new study is planned, in which we plan to use both these environments.